Methods for tagging person identification information to digital data and recommending additional tag by using decision fusion

ABSTRACT

Person identification information is extracted from digital data with a high accuracy and additional tag is recommended by adopting a decision fusion. In other words, the person identification information is acquired by using various additional information to tag the same to the digital data automatically as a person tag, in a digital data management system in which the person identification information is extracted from the digital data by referring to its attributes and contents; and specific digital data having specific tags which are same as those attached to newly created digital data are found out and then candidate tags which are attached to the specific digital data except the specific tags are provided to a monitor in order for a user to choose one or more proper tags from the candidate tags, which are desired to be attached to the newly created digital data additionally, by using the decision fusion.

TECHNICAL FIELD

The present invention relate to a method capable of extracting personidentification information from digital data with a high accuracy and amethod for recommending additional tag with a high precision by adoptinga decision fusion. More particularly, the present invention are directedto a method for acquiring the person identification information by usingvarious additional information more effectively to tag the same to thedigital data automatically as a person tag, in a digital data managementsystem in which the person identification information is extracted fromthe digital data by referring to its attributes and contents and amethod for finding out specific digital data having specific tags whichare same as those attached to newly created digital data and thenproviding candidate tags which are attached to the specific digital dataexcept the specific tags to a monitor of an electronic equipment inorder for a user of the electronic equipment to choose one or moreproper tags from the candidate tags, which are desired to be attached tothe newly created digital data additionally, by using the decisionfusion.

BACKGROUND ART

Recently, the amount of digital data, which are shared by a lot ofpeople, has been tremendously increased, while digital devices such asdigital cameras, camera-attached mobile communication instruments,digital camcorders and MP3 players are popularized. Since digital dataare being frequently produced and widely shared, digital data should besystematically managed. However, it may be difficult to manage digitaldata (e.g., search and information extraction) due to the massive amountof the information therein.

A scheme for classifying or integrating digital data by using a tag iswidely known as one of the conventional techniques for managing digitaldata. A “tag” can be understood as additional data attached to digitaldata for the purpose of accessing or searching for the digital data asquickly as possible. Such a tag is generally comprised of a series ofcharacters, numbers, or a combination of numbers and characters.

There are various types of tags as follows: a space tag, a person tag,an object tag, a time tag and the like. Especially, a lot of attemptshave been made to extract the person tags from digital data with a highaccuracy.

FIG. 7 describes a face recognition by using a face appearance featureand a contextual feature. In this regard, Microsoft Research Asia haspublished an article entitled “Automated Annotation of Human Faces inFamily Albums” in 2003. This article discloses a technique of adding thecontextual feature to the conventional face recognition algorithm inorder to improve a recognition rate of people included in images ofphotos. The contextual feature represents that people included in imagesof a plurality of photos taken on the same day or in the same event maywear the same clothes. Such contextual feature can be used todistinguish each of the people in the photos.

According to the above-mentioned article, the face recognition algorithmbased on the contextual feature cannot be applied to two or more sheetsof photos taken on distant dates, e.g., at intervals of three or moredays. Thus, the article says that the photos are required to be takenwithin at most two days to apply such similarity, i.e., the contextualfeature.

DISCLOSURE OF INVENTION Technical Problem

However, the conventional face recognition technique disclosed in theabove article lacks in accuracy in that the face recognition isperformed by analyzing only the similarity of clothes of people within aspecific range of time and in addition to the similarity of the faceappearance feature. For example, there may often be a case where thesame person wears different clothes even when the specific range of timeis set to one or two days. Specifically, during spring, autumn, orwinter, we can imagine that one who wears a jumper over a shirt in anoutdoor area may wear the shirt merely in an indoor area. Moreover, whena person whose face and body tilted to one side is taken a photographof, there may be a severe problem in that the accuracy of therecognizing rate of the clothes and the face of the person is degraded.

Meanwhile, the prior arts have a drawback in that when a user intends toassign additional tags in addition to tags (for example, at least onetag selected from the space tags, the person tags, the object tags, andthe time tags) given upon creation of new digital data, the user has tomanually input those tags. For example, in a web-based system of Flickr™(http://www.flickr.com) (hereinafter, “flickr”), which is one of theconventional digital data management systems, when a user intends toassign one or more additional tags to digital data (for example,“poppy”) after tagging the digital data as a “dog” while uploading thedigital data, e.g., photos, of a dog to the flickr, the additional tagsshould be inputted manually. Such inconveniences increase as the numberof the additional tags increase.

Technical Solution

It is, therefore, an object of the present invention to provide a methodfor automatically attaching person identification information as aperson tag to digital data by performing a face recognition process witha significantly high accuracy than that of the conventional facerecognition process, to thereby allow a user to manage the digital datamore conveniently and to share the same with other users, by using adecision fusion.

Further, it is another object of the present invention to provide amethod for finding out specific digital data having specific tags whichare same as those attached to newly created digital data and thenproviding candidate tags which are attached to the specific digital dataexcept the specific tags to a monitor of an electronic equipment inorder for a user of the electronic equipment to choose one or moreproper tags from the candidate tags, which are desired to be attached tothe newly created digital data additionally, by using the decisionfusion.

Advantageous Effects

The present invention is capable of extracting the person identificationinformation from the digital data with high accuracy. More specifically,when tags are automatically attached to the digital data based on theextracted person identification information in a digital data managementsystem, the present invention is capable of attaching the personidentification information, i.e., the person tag, to the digital datamore effectively by adopting various additional information on peopleappearing on the digital data.

Moreover, the present invention can significantly reduce the user'sburden of inputting additional tags by perceiving the residual tagswhich are frequently used for the existing digital data including thesame tags as those assigned to the newly created digital data, throughthe use of the decision fusion, thereby assigning one of the residualtags to the digital data.

BRIEF DESCRIPTION OF THE DRAWINGS

The above objects, features and advantages of the present invention willbecome more apparent from the following detailed description when takenin conjunction with the accompanying drawings, in which:

FIG. 1 illustrates an example of improving the accuracy of personrecognition based on the observation that if a specific person appearson any one of photos in a same cluster, the specific person may alsoappear on other photos in the same cluster in accordance with apreferred embodiment of the present invention;

FIG. 2 shows an example of improving the accuracy of the personrecognition based on the observation that a person appearing on a phototaken during a meeting with other person may be a transmitter or areceiver of communication since two or more people tend to communicatewith each other through an e-mail, a telephone, an SMS message or thelike prior to the meeting in accordance with another preferredembodiment of the present invention;

FIGS. 3 and 4 exemplify examples of improving the accuracy of the personrecognition based on the observation that only intimate people out ofall people who have been taken a picture of at least one time mayfrequently appear on the photos in accordance with still anotherpreferred embodiment of the present invention;

FIG. 5 presents an example of making a top-n list through thecombination of results of the person recognition by adopting theBayesian inference in accordance with the present invention;

FIG. 6 illustrates a flowchart for describing a method for recommendingan additional tag in accordance with a further another preferredembodiment of the present invention; and

FIG. 7 depicts a prior art for enhancing the accuracy of the facerecognition.

BEST MODE FOR CARRYING OUT THE INVENTION

The detailed description of the present invention illustrates specificembodiments in which the present invention can be performed withreference to the attached drawings.

In the following detailed description, reference is made to theaccompanying drawings that show, by way of illustration, specificembodiments in which the invention may be practiced. These embodimentsare described in sufficient detail to enable those skilled in the art topractice the invention. It is to be understood that the variousembodiments of the invention, although different, are not necessarilymutually exclusive. For example, a particular feature, structure, orcharacteristic described herein in connection with one embodiment may beimplemented within other embodiments without departing from the spiritand scope of the invention. In addition, it is to be understood that thelocation or arrangement of individual elements within each disclosedembodiment may be modified without departing from the spirit and scopeof the invention. The following detailed description is, therefore, notto be taken in a limiting sense, and the scope of the present inventionis defined only by the appended claims, appropriately interpreted, alongwith the full range of equivalents to which the claims are entitled. Inthe drawings, like numerals refer to the same or similar functionalitythroughout the several views.

The configurations of the present invention for accomplishing theobjects of the present invention are as follows.

In accordance with one aspect of the present invention, there isprovided a method for tagging identification information of a firstperson to a first digital data created by using a digital equipment in adigital data management system, wherein the digital data managementsystem includes both identification information and life patterninformation of a user who creates a plurality of digital data by usingthe digital equipment, and the first digital data includes an image ofthe first person, the method including the steps of: (a) identifying thefirst person by using at least one out of additional information of thefirst person; and (b) automatically attaching the identificationinformation of the first person to the first digital data, wherein theadditional information includes at least one of information: informationwhich indicates that if a first digital data group includes an image ofa second person, a probability that the first person is determined to bethe second person is increased, the first digital data group beingcomprised of the first digital data and other digital data distributedin a same cluster as that including the first digital data, informationwhich indicates that if the first digital data is created while two ormore people have a meeting, a probability that the image of the firstperson is of a transmitter or a receiver of communication made beforethe meeting, information which indicates that if an image of a thirdperson is included m times in a second digital data group, a probabilitythat the first person is the same as the third person increases as thenumber of the m increases, the second digital data group being comprisedof the first digital data and other digital data, the m being a naturalnumber, information associated with the first person acquired by usingat least one of a time when the first digital data is created and aplace where the first digital data is created as well as the lifepattern information of the user of the digital equipment, informationwhich indicates that a probability of the first person being determinedto be a specific person is varied according to a position and a size ofa face of the first person on the image of the first person, informationwhich indicates that if the digital equipment is a dual camera and thefirst digital data is created through a lens of the dual camera directedtoward the user, a probability that the first person is determined to bethe user of the digital equipment is increased, and information whichindicates that if the digital equipment is the dual camera and the firstdigital data is created through a lens of the dual camera directedtoward the opposite side of the user, a probability that the firstperson is determined to be the user of the digital equipment isdecreased.

In accordance with another aspect of the present invention, there isprovided a method for assigning one or more second tags to a firstdigital data additionally, the second tags being different from a firsttag which has been attached to the first digital data, wherein aspecific user creates the first digital data, including at least one ofimages of a specific person P or a specific object O at a specific spaceS in a specific time zone T of a day, by using a digital equipment,wherein at least one of information on the specific space S, thespecific person P, the specific object O, and the specific time zone Thas been attached to the first digital data as the first tag, whereinthe first digital data which is newly created and a plurality ofexisting digital data are stored in a database included in a digitaldata management system, the method including the steps of: (a) searchingidentical_tag_on_digital_data having the same tag as the first tag fromthe existing digital data; (b) calculating how frequently each ofresidual tags, excluding the same tags as the first tag, from all thetags assigned to the identical_tag_on_digital_data, is assigned to theidentical_tag_on_digital_data; (c) displaying candidate tags, having thetop N frequency among the residual tags, on a screen of the digitalequipment; (d) selecting at least one of the candidate tags by the user;and (e) assigning the selected candidate tags to the first digital dataas the second tags of the first digital data.

Mode for the Invention

The First Additional Information for Person Recognition

FIG. 1 illustrates an example of improving the accuracy of personrecognition based on the observation that if a specific person appearson any one of photos in a same cluster, the specific person may alsoappear on the other photos in the same cluster in accordance with apreferred embodiment of the present invention. The cluster may bepartitioned by referring to the time when digital data such as digitalphotos are created or the place where the digital data are created.

The concept of the cluster is also mentioned in the section of “clustertagging” included in the detailed description of Korean PatentApplication No. 2006-14040 filed by the applicant of the presentinvention. According to it, the digital data may be created by using adigital device at any time and at any place. However, due to thediscontinuity of the time when the digital data is created or the placewhere the digital data is generated, the digital data shows adiscontinuous distribution. It is derived from the observation that, ina daily life, the digital data tends to be produced only in certain timeand place by a user. Therefore, the cluster tagging for clusteringrelevant digital data into one cluster by acquiring the place or thetime pertaining to the various digital data, and assigning a common tagto all the relevant digital data may be adopted.

The person tag can be extracted with a high accuracy by applying theconcept of the cluster tagging as set forth above in accordance withpresent invention. In other words, if a plurality of digital data arecreated within a same cluster and an image of a specific person isincluded in one of the plurality of the digital data, the image of thespecific person may also appear on the other photos in the same cluster.More specifically, it is based on the observation that a plurality ofdigital data may be created for a specific person with different poseswithin a range of a specific time, and moreover, plenty of digital datamay be created for the specific person with different backgrounds withina range of a specific place. In other words, it may be the generaltendency of people.

Accordingly, the accuracy of the recognition rate can be significantlyenhanced by using the tendency that the specific person may repeatedlyappear within the same cluster, thereby assigning an appropriate persontag to the digital data created at the ranges of the specific time ofplace.

Specifically, a list of candidates who are determined to be appeared topn frequently on photos taken within a cluster is created by analyzingthe frequency of appearance of people in the images of digital datawhich have already been created within the same cluster. Thereafter, ifdigital data including an image of a specific person is newly created inthe same cluster, one of people who have been assigned the highest top nprobability, i.e., the highest top n frequency of appearance, among thelist of candidates which has already been created may be determined tobe the specific person of the newly created digital data. Then, if theidentity of the specific person appearing on the image of the newlycreated digital data is determined, it is possible to update aprobability value assigned to each person included in the list ofcandidates in real-time. This process may be performed by a decisionfusion, e.g., a Bayesian inference, which will be described later.

The Second Additional Information for Person Recognition

FIG. 2 shows an example of improving the accuracy of the personrecognition based on the observation that a person appearing on a phototaken during a meeting with other person may be a transmitter or areceiver of communication since two or more people tend to communicatewith each other through an e-mail, a telephone, an SMS message, achatting service (e.g., MSN messenger) or the like prior to the meetingin accordance with another preferred embodiment of the presentinvention.

As shown in the example of message communication of FIG. 2, peoplegenerally tend to communicate with each other to make a decision on aplace or a time for a meeting, prior to the meeting. Such acommunication may be performed through the e-mail, the telephone, theSMS message using a cellular phone, and the chatting service. A detailedexample of message communication shown in FIG. 2 is as follows. That is,it can be seen that two users, e.g., Jiro and Tanaka, have communicatedwith each other to make a decision on the time when they meet throughthe SMS message, and then actually, photos on which both Jiro and Tanakaappear were created after the communication. Thus, the accuracy ofrecognition on people appearing on photos can be improved by fullyconsidering such correlation.

The above-described additional information can be applied as a verystrong tool since it is considered to be the general tendency of people.In detail, two or more people apart from each other are required to meetat a specific place after a fixed time in order to take photos together.In order for them to meet at the specific place, the communication mustbe made between them to make a decision on a place and a time for themeeting. By using such a general tendency of people properly, theaccuracy of recognition on people appearing on images of digital datacan be significantly enhanced.

Specifically, a list of candidates may be created after finding out thetransmitter and the receiver of the communication made during a certaintime before digital data is newly created, as set forth above.Thereafter, if the digital data including an image of a specific personis newly created, one of people who have been assigned the highest top nprobability among the list of candidates which has already been createdmay be determined to be the specific person of the newly created digitaldata. Then, if the identity of the specific person appearing on theimage of the newly created digital data is determined, it is possible toupdate a probability value assigned to each person included in the listof candidates in real-time. Afterwards, if another digital dataincluding an image of a certain person is newly created, the identity ofthe certain person is determined on the basis of the list of candidateswhich has been updated. This process may be conducted by the decisionfusion, e.g., the Bayesian inference, which will be described later.

The Third Additional Information for Person Recognition

FIGS. 3 and 4 exemplify examples of improving the accuracy of the personrecognition based on the observation that only intimate people out ofall people who have been taken a picture of at least one time mayfrequently appear on the photos in accordance with still anotherpreferred embodiment of the present invention.

In FIG. 3, x-axis represents a rank of people who has appeared on photosat least one time, the rank being determined according to the frequencyof the appearance of each person in descending order, while the totalnumber of individual persons appearing on the photos is set to be 100,and y-axis represents a percentage of photos while the total number ofphotos corresponds to 100%. Arbitrary coordinate values (X and Y) on thegraph depicted on an xy plane represent that the number of photos wheretop X persons appear corresponds to Y% in total.

For example, as shown in FIG. 3, it can be seen that the number ofphotos where top 9 persons appear occupies 37% of the total number ofphotos. This indicates that the 9 persons only occupy 9% of the totalnumber of persons (i.e., 100 persons) on photos, but the percentageoccupied by the 9% of the total number of persons reaches up to 37% ofthe total number of photos. As shown in FIG. 3, it can be found that thegraph is highly tilted toward left-upper side.

If a graph were directly proportional in the xy-plane, it wouldrepresent that all people appear on the photos at the identicalfrequency, which means that there is no useful information for the facerecognition. However, people actually tend to take photos with theirintimate ones, so that if a graph is highly deviated from a directlyproportional graph, as shown in FIG. 3, it has an extremely valuableinformation for the face recognition, thereby enhancing the accuracy ofthe recognition of people appearing on photos.

FIG. 4 is a bar graph (X, Y) in which x-axis represents the number ofphotos and y-axis represents the number of persons. Herein, the bargraph (X, Y) indicates that Y-number of persons have X-number of photoson which their own faces appear. For example, a bar where X=1 and Y=24means that 24 persons have 1 sheet of photo, respectively, where theirown faces appear. Further, another bar where X=40 and Y=1 representsthat 1 person has as many as 40 sheets of photos where his or her ownface appears.

There are three boxes with dotted line as shown in FIG. 4. The rightmostbox indicates people whose photos are taken at the highest frequency, sothat the photos included in the rightmost box may be self-shots orfamily photographs. Moreover, the middle box shows people whose photosare taken at the second highest frequency, so that the photos includedin the middle box may contain images of best friends. Furthermore, theleftmost box shows people whose photos are taken at the third highestfrequency, so that the photos included in the leftmost box may containimages of intimate friends. Finally, photos belonging to bar graphslocated at left side of the leftmost box may contain images ofacquaintance. In this regard, there may be naturally a personaldifference. The personal difference can be applied by updating the listof candidates as mentioned above. For example, photos belonging to therightmost box may be of best friends rather than of self-shot or familymembers according to the personal difference. The identity of peopleincluded in each box can be modified by using the feedback from thehuman relationship, i.e., the feedback from the analysis of thefrequency of appearance of person on photos previously taken, so thatthe accuracy of face recognition on people appearing on photos can beenhanced.

In short, a list of candidates is created by analyzing top n personsappearing on photos, i.e., digital data, which are already created, atthe highest frequency. Thereafter, if the digital data including animage of a specific person is newly created, one of people who have beenassigned the highest top n probability among the list of candidateswhich has already been created may be determined to be the specificperson of the newly created digital data.

Then, if the identity of the specific person appearing on the image ofthe newly created digital data is determined, it is possible to update aprobability value assigned to each person included in the list ofcandidates in real-time. Afterwards, if another digital data includingan image of a certain person is newly created, the identity of thecertain person is determined on the basis of the list of candidateswhich has been updated. This process may be performed by the decisionfusion, e.g., the Bayesian inference, which will be described later.

The Fourth Additional Information for Person Recognition

In order to enhance the accuracy of recognition on faces of peopleappearing on photos, i.e., digital data, one or more tags eitherautomatically or manually attached to the digital data can be referredto. For example, space tags or time tags either automatically ormanually attached to the digital data may be referred to. Specifically,the recognition on certain people appearing on the digital data can beperformed by referring to the tags attached to the digital data inaddition to life patterns of the certain people.

For instance, if photos, i.e., digital data, including the certainpeople are taken during a weekend (that is, time tags of the digitaldata indicate the weekend), weight may be set to the probability thatthe certain people on the photos are family members or friends; if thephotos are taken during weekdays (that is, the time tags of the digitaldata indicate weekdays), the weight may be set to the probability thatthe certain people on the photos are fellow workers; if the photos aretaken at a pub (that is, space tags of the digital data indicate thepub), the weight may be set to the probability that the certain peopleon the photos are friends or fellow workers; and if the photos are takenat home (that is, the space tags of the digital data indicate home), theweight may be set to the probability that the certain people on thephotos are family members.

In the same way, even if the certain people have peculiar life patterns,for example, if the certain people generally have a meeting with theirfriends at an amusement park rather than at a pub, or if the certainpeople enjoy solitude rather than being with family members or friendsduring weekends, the peculiar life patterns can be automaticallyanalyzed by analyzing the existing photos, thereby modifying the weightof the probability according thereto.

In summary, the life patterns of the certain people appearing on imagesof the existing digital data are analyzed to create a list ofcandidates, the list including top n persons who are assigned a highprobability of being determined to be specific people appearing onimages of digital data which are going to be newly created at a specifictime and in a specific place. Thereafter, if the digital data includingthe images of the specific people are newly created, one of people whohave been assigned the highest top n probability among the list ofcandidates which has already been created may be determined to be thespecific people of the newly created digital data.

Then, if the identities of the specific people appearing on the imagesof the newly created digital data are determined, it is possible toupdate a probability value assigned to each person included in the listof candidates in real-time. Afterwards, if another digital dataincluding an image of a particular person is newly created, the identityof the particular person is determined on the basis of the list ofcandidates which has been updated. This process may be performed by thedecision fusion, e.g., the Bayesian analysis, which will be describedlater.

The Fifth Additional Information for Person Recognition

The accuracy of face recognition can be enhanced by analyzing thelocations and the sizes of one or more faces appearing on a photo. Forexample, if one large-sized face of a specific person appears at thecenter of a photo, the photo may be a self-shot of the specific person;and if one large-sized face of the specific person appears at the centerof the photo and very small-sized faces appear around the large-sizedface, the large-sized face at the center of the photo may be a self-shotof the specific person and the very small-sized faces may be strangers,e.g., walkers. Further, if two faces of similar sizes appear on thephoto and one of the two faces is already tagged as “me”, it is likelythat the identity of the other face is my best friend or my girl friend.As mentioned above, it is possible to update the probability valueassigned to each person included in the list of candidates in real-timeby analyzing existing photos, e.g., general tendencies or peculiarhabits of people appearing on the existing photos. This process can becarried out with reference to the decision fusion, e.g., the Bayesianinference, which will be described later.

Besides, there may be a lot of methods for enhancing the accuracy of theface recognition. For instance, in case of a dual camera, an identity ofa person on a photo taken through one lens directed toward a user of thecamera may be the user himself or herself; and the identity of theperson on the photo taken through the other lens directed toward theopposite side of the user may be a friend of the user. Under theassumption that the self-shot occupy a considerable part of all thephotos, the accuracy of the face recognition can be enhanced because theself-shot is excluded if the other lens of the camera directed towardthe opposite side of the user is used.

Combination of Additional Information for Person Recognition ThroughDecision Fusion

FIG. 5 illustrates a process of making a top-n list through thecombination of cues in accordance with the above-described embodimentsby the decision fusion, e.g., the Bayesian inference, wherein the top-nlist includes top n persons who are assigned high probabilities of beingdetermined to be a specific person included in images of newly createddigital data, the n being a natural number, e.g., 9.

The decision fusion is a recursive process for combining informationwhich is accumulated as time goes by in order to reply to somescientific questions. Specifically, the decision fusion is an analyticalmethod for evaluating the current status of a subject, i.e., an issue,collecting new data to answer the questions about the subject, andupdating the knowledge about the subject by combining the collected newdata and the existing data, thereby reducing errors. In short, thedecision fusion represents a process for making a decision on a specificissue by using multiple cues and a priori probability.

The decision fusion includes various methods such as an ad-hoc method,the Bayesian inference, etc. The ad-hoc method calculates probabilitiesfrom each of the cues and obtains a final probability by summing up thecalculated probabilities, e.g., by using “weighted probability sum” tocreate a torn list Unlike the ad-hoc method, the Bayesian inferencecreates a torn list by calculating a probability that takes into accountall the cues. In general, the cues are not independent from each other.For example, there exists a case where a time factor and a space factorare correlated. Specifically, an office worker probably stays in his/heroffice (the space factor) during daytime of weekdays (the time factor).Since the cues are not independent as mentioned above, the ad-hoc methodmay not be optimal. Therefore, the Bayesian inference is preferredrather than the ad-hoc method in accordance with the present invention.However, it should be noted that the ad-hoc method can be adopted toembody the present invention in certain cases.

By using the Bayesian inference, a top-9 list can be created bycombining additional information which was obtained from theabove-described embodiments. A box on the left side shown in FIG. 5indicates a probability of being determined to be a specific person ondigital data which are going to be newly created, for each of 100persons, under the assumption that the total number of persons on allthe existing photos is 100. Herein, the probability is calculated byusing all of the above-mentioned additional information for the facerecognition. The probability written in the box on the left side shownin FIG. 5 is updated whenever digital data is newly created, which iscaused by the attributes of the Bayesian inference. Afterwards, ifanother digital data including images of a certain person are newlycreated, an identity of the certain person may be determined on thebasis of the updated probability. In other words, the probability isdynamically modified, thereby achieving more accurate estimation. As aresult, the top-9 list shown in a box on the right side of FIG. 5 can becreated by using the Bayesian inference. By referring to the createdtop-9 list, errors that might be caused during the face recognitionprocess can be significantly removed.

Additional Tag Recommendation Function

FIG. 6 illustrates a flowchart describing a method for recommending oneor more additional tags in accordance with a further another preferredembodiment of the present invention.

First of all, digital data, including at least one of images of aspecific person or a specific object at a specific time zone during oneday at a specific place, are newly created by using a digital equipmentby a specific user, while a plurality of existing digital data arestored in a database of a digital data management system (step 601). Thenewly created digital data have one or more tags including informationon at least one of a space, a person, an object, and a time zone, andthe tags are automatically assigned to the digital data at the time ofthe creation thereof (see, Korean Patent Application No. 2006-14040filed by the same applicant as that of the present invention).

Thereafter, digital data having the same tags as those automaticallyassigned to the newly created digital data are searched from thedatabase (step 602). Herein, specific tags, excluding the same tagsdescribed in the step 602, attached to the searched digital data arecalled residual tags.

Then, how frequently each of the residual tags has been assigned to thesearched digital data is calculated (step 603).

Subsequently, candidate tags having the top N frequency among theresidual tags are displayed on a screen of the digital equipment for theuser's selection (step 604), and one or more tags are selected by theuser from the candidate tags displayed on the screen (step 605).

Accordingly, additional tags, i.e., the selected tags, are attached tothe newly created digital data (step 606).

For example, it is likely that a specific user frequently attaches a tagcalled “shopping” or “date” to digital data if the specific user and hisgirl friend often shop at a specific department store when they takephotos together. As can be seen from the above-mentioned Korean PatentApplication No. 2006-14040, the digital data created by using thedigital equipment by the specific user may be automatically tagged withthe “specific department store” by using GPS for tracking the positionof the digital equipment. Further, a name of the girl friend may beautomatically tagged through face recognition on a person appearing onthe digital data in accordance with the present invention. In this case,if existing digital data having the same tags as those attached to thenewly created digital data automatically, e.g., the specific departmentstore or the name of the girl friend, are searched from the database,tags attached to the searched digital data with a high frequency may betags of “shopping” or “date”. By calculating the rank of the frequencythrough the decision fusion, e.g., the Bayesian inference, a top-n listincluding tags with top n frequency, e.g., a top-9 list can be created.The top-n list is displayed on the screen of the digital equipment sothat the user can select any of them. Accordingly, the present inventioncan implement a user-friend tag inputting method, which cannot berealized by the prior arts such as the flickr and the like.

Meanwhile, the top-n list is updated in real-time through the Bayesianinference; and therefore, if another digital data is newly created, tagsof just previously created digital data are also reflected so that theuser can select additional tags from the updated top-n list.

While the invention has been shown and described with respect to thepreferred embodiments, it will be understood by those skilled in the artthat various changes and modification may be made without departing fromthe spirit and scope of the invention as defined in the followingclaims.

1. A method for tagging identification information of a first person toa first digital data created by using a digital equipment in a digitaldata management system, wherein the digital data management systemincludes both identification information and life pattern information ofa user who creates a plurality of digital data by using the digitalequipment, and the first digital data includes an image of the firstperson, the method comprising the steps of: (a) identifying the firstperson by using at least one out of additional information of the firstperson; and (b) automatically attaching the identification informationof the first person to the first digital data, wherein the additionalinformation includes at least one of information: information whichindicates that if a first digital data group includes an image of asecond person, a probability that the first person is determined to bethe second person is increased, the first digital data group beingcomprised of the first digital data and other digital data distributedin a same cluster as that including the first digital data, informationwhich indicates that if the first digital data is created while two ormore people have a meeting, a probability that the image of the firstperson is of a transmitter or a receiver of communication made beforethe meeting is increased, information which indicates that if an imageof a third person is included m times in a second digital data group, aprobability that the first person is the same as the third personincreases as the number of the m increases, the second digital datagroup being comprised of the first digital data and other digital data,the m being a natural number, information associated with the firstperson acquired by using at least one of a time when the first digitaldata is created and a place where the first digital data is created aswell as the life pattern information of the user of the digitalequipment, information which indicates that a probability of the firstperson being determined to be a specific person is varied according to aposition and a size of a face of the first person on the image of thefirst person, information which indicates that if the digital equipmentis a dual camera and the first digital data is created through a lens ofthe dual camera directed toward the user, a probability that the firstperson is determined to be the user of the digital equipment isincreased, and information which indicates that if the digital equipmentis the dual camera and the first digital data is created through a lensof the dual camera directed toward the opposite side of the user, aprobability that the first person is determined to be the user of thedigital equipment is decreased.
 2. The method of claim 1, wherein thestep (a) includes a step of combining the additional information by adecision fusion, the decision fusing being a general term of a processfor making a decision on a specific issue from multiple cues and apriori probability.
 3. The method of claim 2, wherein the decisionfusion includes Bayesian inference or ad-hoc analysis.
 4. The method ofclaim 3, wherein the step (a) further includes a step of creating atop-n list, in which candidates having the top n probability of beingdetermined to be the first person are included, based on the result ofthe combination of the additional information through the Bayesianinference or the ad-hoc analysis.
 5. The method of claim 4, wherein thetop-n list is updated in real-time through the Bayesian inference or thead-hoc analysis.
 6. The method of claim 4 or 5, wherein the step (b)includes a step of determining the first person to be a person havingthe highest probability among the candidates of top-n list.
 7. Themethod of claim 1, wherein the communication is at least one of e-mailcommunication, wire or wireless telephone communication, SMS messagecommunication, and messenger communication.
 8. The method of claim 1,wherein the information on the receiver or the transmitter of thecommunication made before the meeting is acquired from a communicationdevice used for the meeting.
 9. The method of claim 1, wherein theinformation on the receiver or the transmitter of the communication madebefore the meeting is acquired by recognizing contents of thecommunication.
 10. The method of claim 1, wherein the informationassociated with the first person acquired by using at least one of thetime when the first digital data is created and the place where thefirst digital data is created as well as the life pattern information ofthe user of the digital equipment is: information which indicates that aprobability of the first person being determined to be a family memberor a friend of the user is increased if the time corresponds to aholiday, and information which indicates that a probability of the firstperson being determined to be a fellow worker of the user is increasedif the time corresponds to a weekday.
 11. The method of claim 1, whereinthe information associated with the first person acquired by using atleast one of the time when the first digital data is created and theplace where the first digital data is created as well as the lifepattern information of the user of the digital equipment is: informationwhich indicates that a probability of the first person being determinedto be a friend or a fellow worker of the user is increased if the placeis a drinking house or a bar, and information which indicates that aprobability of the first person being determined to be a family memberof the user is increased if the place is home.
 12. The method of claim11, wherein the place is perceived by referring to tags attached to thefirst digital data.
 13. The method of claim 1, wherein if the firstdigital data further includes an image of a fourth person, theinformation which indicates that a probability of the first person beingdetermined to be a specific person is varied according to a position anda size of a face of the first person on the image of the first personis: information which indicates that a probability of the first personbeing determined to be the specific person is decreased if a face of thefourth person is positioned at the center of the whole image of thefirst digital data and the face of the fourth person is remarkablylarger than that of the first person.
 14. The method of claim 1, whereinif the first digital data further includes an image of a fifth person,the information which indicates a probability of the first person beingdetermined to be a specific person is varied according to a position anda size of a face of the first person on the image of the first personis: information which indicates that a probability of the first personbeing determined to be a friend of a fifth person is increased if a sizeof the face of the first person is similar to that of a face of thefifth person.
 15. The method of claim 14, wherein the fifth person isidentified by referring to tags attached to the first digital data. 16.A method for assigning one or more second tags to a first digital dataadditionally, the second tags being different from a first tag which hasbeen attached to the first digital data, wherein a specific user createsthe first digital data, including at least one of images of a specificperson P or a specific object O at a specific space S in a specific timezone T of a day, by using a digital equipment, wherein at least one ofinformation on the specific space S, the specific person P, the specificobject O, and the specific time zone T has been attached to the firstdigital data as the first tag, wherein the first digital data which isnewly created and a plurality of existing digital data are stored in adatabase included in a digital data management system, the methodcomprising the steps of: (a) searching identical_tag_on_digital_datahaving the same tag as the first tag from the existing digital data; (b)calculating how frequently each of residual tags, excluding the sametags as the first tag, from all the tags assigned to theidentical_tag_on_digital_data, is assigned to theidentical_tag_on_digital_data; (c) displaying candidate tags, having thetop N frequency among the residual tags, on a screen of the digitalequipment; (d) selecting at least one of the candidate tags by the user;and (e) assigning the selected candidate tags to the first digital dataas the second tags of the first digital data.
 17. The method of claim16, wherein the frequency is calculated by a decision fusion, thedecision fusion being a general term of a process for making a decisionon a specific issue from multiple cues and a priori probability.
 18. Themethod of claim 17, wherein the decision fusion includes Bayesianinference or ad-hoc analysis.
 19. The method of claim 18, wherein thecandidate tags are updated in real-time.